Impaired value-based decision-making in Parkinson's disease apathy.

Apathy is a common and disabling complication of Parkinson's disease characterized by reduced goal-directed behaviour. Several studies have reported dysfunction within prefrontal cortical regions and projections from brainstem nuclei whose neuromodulators include dopamine, serotonin and noradrenaline. Work in animal and human neuroscience have confirmed contributions of these neuromodulators on aspects of motivated decision-making. Specifically, these neuromodulators have overlapping contributions to encoding the value of decisions, and influence whether to explore alternative courses of action or persist in an existing strategy to achieve a rewarding goal. Building upon this work, we hypothesized that apathy in Parkinson's disease should be associated with an impairment in value-based learning. Using a four-armed restless bandit reinforcement learning task, we studied decision-making in 75 volunteers; 53 patients with Parkinson's disease, with and without clinical apathy, and 22 age-matched healthy control subjects. Patients with apathy exhibited impaired ability to choose the highest value bandit. Task performance predicted an individual patient's apathy severity measured using the Lille Apathy Rating Scale (R = -0.46, P < 0.001). Computational modelling of the patient's choices confirmed the apathy group made decisions that were indifferent to the learnt value of the options, consistent with previous reports of reward insensitivity. Further analysis demonstrated a shift away from exploiting the highest value option and a reduction in perseveration, which also correlated with apathy scores (R = -0.5, P < 0.001). We went on to acquire functional MRI in 59 volunteers; a group of 19 patients with and 20 without apathy and 20 age-matched controls performing the Restless Bandit Task. Analysis of the functional MRI signal at the point of reward feedback confirmed diminished signal within ventromedial prefrontal cortex in Parkinson's disease, which was more marked in apathy, but not predictive of their individual apathy severity. Using a model-based categorization of choice type, decisions to explore lower value bandits in the apathy group activated prefrontal cortex to a similar degree to the age-matched controls. In contrast, Parkinson's patients without apathy demonstrated significantly increased activation across a distributed thalamo-cortical network. Enhanced activity in the thalamus predicted individual apathy severity across both patient groups and exhibited functional connectivity with dorsal anterior cingulate cortex and anterior insula. Given that task performance in patients without apathy was no different to the age-matched control subjects, we interpret the recruitment of this network as a possible compensatory mechanism, which compensates against symptomatic manifestation of apathy in Parkinson's disease.

Pallidal neuromodulation of the explore/exploit trade-off in decision-making.

Every decision that we make involves a conflict between exploiting our current knowledge of an action's value or exploring alternative courses of action that might lead to a better, or worse outcome. The sub-cortical nuclei that make up the basal ganglia have been proposed as a neural circuit that may contribute to resolving this explore-exploit 'dilemma'. To test this hypothesis, we examined the effects of neuromodulating the basal ganglia's output nucleus, the globus pallidus interna, in patients who had undergone deep brain stimulation (DBS) for isolated dystonia. Neuromodulation enhanced the number of exploratory choices to the lower value option in a two-armed bandit probabilistic reversal-learning task. Enhanced exploration was explained by a reduction in the rate of evidence accumulation (drift rate) in a reinforcement learning drift diffusion model. We estimated the functional connectivity profile between the stimulating DBS electrode and the rest of the brain using a normative functional connectome derived from heathy controls. Variation in the extent of neuromodulation induced exploration between patients was associated with functional connectivity from the stimulation electrode site to a distributed brain functional network. We conclude that the basal ganglia's output nucleus, the globus pallidus interna, can adaptively modify decision choice when faced with the dilemma to explore or exploit.

Tonic dopamine, uncertainty and basal ganglia action selection.

To make optimal decisions in uncertain circumstances flexible adaption of behaviour is required; exploring alternatives when the best choice is unknown, exploiting what is known when that is best. Using a computational model of the basal ganglia, we propose that switches between exploratory and exploitative decisions are mediated by the interaction between tonic dopamine and cortical input to the basal ganglia. We show that a biologically detailed action selection circuit model, endowed with dopamine dependant striatal plasticity, can optimally solve the explore-exploit problem, estimating the true underlying state of a noisy Gaussian diffusion process. Critical to the model's performance was a fluctuating level of tonic dopamine which increased under conditions of uncertainty. With an optimal range of tonic dopamine, explore-exploit decisions were mediated by the effects of tonic dopamine on the precision of the model action selection mechanism. Under conditions of uncertain reward pay-out, the model's reduced selectivity allowed disinhibition of multiple alternative actions to be explored at random. Conversely, when uncertainly about reward pay-out was low, enhanced selectivity of the action selection circuit facilitated exploitation of the high value choice. Model performance was at the level of a Kalman filter which provides an optimal solution for the task. These simulations support the idea that this subcortical neural circuit may have evolved to facilitate decision making in non-stationary reward environments. The model generates several experimental predictions with relevance to abnormal decision making in neuropsychiatric and neurological disease.

Opposing patterns of abnormal D1 and D2 receptor dependent cortico-striatal plasticity explain increased risk taking in patients with DYT1 dystonia.

Patients with DYT1 dystonia caused by the mutated TOR1A gene exhibit risk neutral behaviour compared to controls who are risk averse in the same reinforcement learning task. It is unclear whether this behaviour can be linked to changes in cortico-striatal plasticity demonstrated in animal models which share the same TOR1A mutation. We hypothesised that we could reproduce the experimental risk taking behaviour using a model of the basal ganglia under conditions where cortico-striatal plasticity was abnormal. As dopamine exerts opposing effects on cortico-striatal plasticity via different receptors expressed on medium spiny neurons (MSN) of the direct (D1R dominant, dMSNs) and indirect (D2R dominant, iMSNs) pathways, we tested whether abnormalities in cortico-striatal plasticity in one or both of these pathways could explain the patient's behaviour. Our model could generate simulated behaviour indistinguishable from patients when cortico-striatal plasticity was abnormal in both dMSNs and iMSNs in opposite directions. The risk neutral behaviour of the patients was replicated when increased cortico-striatal long term potentiation in dMSN's was in combination with increased long term depression in iMSN's. This result is consistent with previous observations in rodent models of increased cortico-striatal plasticity at in dMSNs, but contrasts with the pattern reported in vitro of dopamine D2 receptor dependant increases in cortico-striatal LTP and loss of LTD at iMSNs. These results suggest that additional factors in patients who manifest motor symptoms may lead to divergent effects on D2 receptor dependant cortico-striatal plasticity that are not apparent in rodent models of this disease.

Maladaptive striatal plasticity and abnormal reward-learning in cervical dystonia.

In monogenetic generalized forms of dystonia, in vitro neurophysiological recordings have demonstrated direct evidence for abnormal plasticity at the level of the cortico-striatal synapse. It is unclear whether similar abnormalities contribute to the pathophysiology of cervical dystonia, the most common type of focal dystonia. We investigated whether abnormal cortico-striatal synaptic plasticity contributes to abnormal reward-learning behavior in patients with focal dystonia. Forty patients and 40 controls performed a reward gain and loss avoidance reversal learning task. Participant's behavior was fitted to a computational model of the basal ganglia incorporating detailed cortico-striatal synaptic learning rules. Model comparisons were performed to assess the ability of four hypothesized receptor specific abnormalities of cortico-striatal long-term potentiation (LTP) and long-term depression (LTD): increased or decreased D1:LTP/LTD and increased or decreased D2: LTP/LTD to explain abnormal behavior in patients. Patients were selectively impaired in the post-reversal phase of the reward task. Individual learning rates in the reward reversal task correlated with the severity of the patient's motor symptoms. A model of the striatum with decreased D2:LTP/ LTD best explained the patient's behavior, suggesting excessive D2 cortico-striatal synaptic depotentiation could underpin biased reward-learning in patients with cervical dystonia. Reversal learning impairment in cervical dystonia may be a behavioral correlate of D2-specific abnormalities in cortico-striatal synaptic plasticity. Reinforcement learning tasks with computational modeling could allow the identification of molecular targets for novel treatments based on their ability to restore normal reward-learning behavior in these patients.

Estimation of the phase response curve from Parkinsonian tremor.

Phase response curves (PRCs), characterizing the response of an oscillator to weak external perturbation, have been estimated from a broad range of biological oscillators, including single neurons in vivo. PRC estimates, in turn, provide an intuitive insight into how oscillatory systems become entrained and how they can be desynchronized. Here, we explore the application of PRC theory to the case of Parkinsonian tremor. Initial attempts to establish a causal effect of subthreshold transcranial magnetic stimulation applied to primary motor cortex on the filtered tremor phase were unsuccessful. We explored the possible explanations of this and demonstrate that assumptions made when estimating the PRC in a traditional setting, such as a single neuron, are not arbitrary when applied to the case of tremor PRC estimation. We go on to extract the PRC of Parkinsonian tremor using an iterative method that requires varying the definition of the tremor cycle and estimating the PRC at multiple peristimulus time samples. Justification for this method is supported by estimates of PRC from simulated single neuron data. We provide an approach to estimating confidence limits for tremor PRC and discuss the interpretational caveats introduced by tremor harmonics and the intrinsic variability of the tremor's period.